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ABSTRACT 

The fact that college grades are poor predictors of 
future job performance is a cause for concern. A more important issue 
is assessment, for course grades cannot measure many of the 
work-relevant skills that a college education provides. Selection 
tests are one effort to identify and establish those characteristics 
of applicants that predict future job success. If classroom grades 
could be broken down to reveal performance in such areas as verbal 
ability or memory, they would be indicative of subsequent job 
performance because they would essentially duplicate ability tests; 
ability tests, however, are typically three to four times better at 
predicting job performance. Another way to obtain information on a 
job candidate is through what is called "bio-data"; detailed 
information on extracurricular activities may reveal knowledge, 
skills, and abilities (KSAs) acquired by a student outside 
traditional classroom settings. Many colleges experiences provide 
what is the equivalent of work sample tests. Another way to get at 
the question of what predicts job success is to look directly at the 
requirements of jobs . Job analysis refers to systematic efforts to 
collect information about the work requirements associated with 
particular jobs. Job analyses either focus descriptions on the job 
and tasks performed or are written from the perspective of the worker 
and describe the KSAs required. The basic sets of KSAs could be 
developed more thoroughly in college instruction. The greatest 
improvements in assessment could be made by simply assembling 
existing information about student performance in more innovative 
ways. An appendix provides descriptions of some of the most widely 
used job analysis systems and identifies the KSAs that are stressed 
in them. (Contains 17 endnotes and 32 references ) (YLB) 
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I. Introduction 



The rising cost of higher education and serious concerns 
about the relationship between student learning and the 
strength of the national economy have made the perfor- 
mance of college students an issue for public policy. One 
objective set by the National Education Goals Panel (NEGP) 
is to develop a method of assessing the performance of 
college graduates, a task that reflects a growing concern that 
college students are not learning "enough" for the good of 
the nation. Disenchantment with traditional assessment 
methods already has caused a wave of experimentation 
among secondan- schools' — the time has come for similar 
experiments at the college level. 



The related NEGP goal of strengthening the link between 
education and the workplace can also be served by an 
assessment of student performance. The experience that the 
business community has accrued in identifying the skills 
required for college-level jobs and in testing employees to 
predict their employment success can pla} a valuable role in 
the effort. While improving job performance is just one of 
many important goals for education, it is increasingly 
important because we can now see more clearly the strong 
links between individual job performance and the national 
economy. The evidence suggests that education at the 
college level may not be serving this goal — and that doing so 
would not impinge on other important objectives of education. 
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II. The Relationship Between Educational and Job Performance 



In order to determine the relationship between 
educational performance' and job performance, two initial 
questions must be considered: Vi'hat effect does one's 
educational performance have on job performance? \^ hat 
aspects of education are most important in producing 
effective employees? 

John Bishop surveyed economics research and concluded 
that performance in high school explains almost nothing 
about job success. Although a high school di[)loma is an 
important credential, superior performance in school, when 
defined as earning high grades, does not improve the 
probability of getting a job nor of receiving higher wages 
once one has a job (Bishop 1989), Most of this research is 
based on data from the National l/)ngitudinal Sample (NLS), 
which tracked over time a cohort of high school graduates. 
On the college level, the NLS-based research on the rela- 
tionship between performance in college and on the job has 
been complicated by difficulties in coding transcript data, 
and the results are not always consistent. College grades 
from this data set, however, also do not appear to be good 
predictors of sei'uring jobs or of the level of wages. 

The other area of research on th(^ subject, performed 
mainly by industrial psychologists, is more extensive and 
typically uses actual measures of job perfornance (as 
opposed to indirect measun^s: e.g., wages) to measure* 
worker success. In this field, it has been argued for decades 
that college grades are not good predictors of actual perfor- 
mance on the job (McClelland 1973). Although some 



studies do find a relationship between grades and measures 
of job performance within individual firms, it is remarkable 
just how many published studies re[)ort no relationship. 
This is especially surprising given that it is not a common 
practice of researchers to publish studies that show an 
(ibsence of significant relationships and that journals typi- 
callv are not interested in such reports.^ Rol)ert Brelz 
(1989) performed one of the more recent meta-analyses/ 
statistical summaries of previous grade point average 
research, using a large sample of studies, and found no 
overall relationship with adult achievement in the 
workplace. David Dye and Martin Reck (1989) used a 
similar sample of studies. They controlled for sampling 
error and unreliability of criterion measure in the original 
series through a series of corrections. They reported a 
significant, albeit ver\ small, validity coefficient of 0.18. 
(Validitv coefficients are correlations, and iheir square is the 
coefficient of determination — the popular R2 measure. Dye 
and Reek's 0.18 correlation, for example, means that grades 
explain 0.032 percent of the variance in job performance 
measures, which is not a lot.)* 

Kven the proponents of using grades as [)rcdictors claim 
onlv that grades ha\e .w?»e relevance, and not that they are 
powerful predictors in the absolute sense. It is easy to find 
studies in almost every context that find no relationship 
between grade* point average and perfornumce: in business, 
using overall m<*asures of job success f'\g., Ferris 1982); in 
engineering (e.g.. Muchinsky and Hoyt 1973); and in 
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performance in graduate programs (e.g., Harrell and Harrell 
1984. cited in Howard 1986). Even the quality of the 
college, measured by its selectivity in admissions, has been 
shown to add little to predictions of job performance (e.g., 
Ferris 1982; Howard 1986: Rosenbaum 1984). Perhaps the 
most successful attempt to find a relationship between 
grades and job performance was the study at AT&T by 
Howard (1986). The project used employee-ability data 
from assessment centers (which created simulations to proxy 
job performance) as an indirect measure of job performance. 
Howard found that undergraduate grades have statistically 
significant relationships with only about one-fourth of the 
measures of job performance and potential. 

Reviewers of all categories of selection procedures, such 
as Reilly and C\n\o (1982) and Hunter and Hunter (1984). 
assert that grades, in ternv.^ of their predictive power, are 
well toward the bottom of any ranking of selection devices. 
Although grades may help fleterniine one's occupation and 
income (largely through affecting admission to graduate 
schools), and graduates who receive a college degree (as 
opposed to simply taking courses) usually earn significantly 
more, grades and their indication of performance in college 
are poorly related to ac tual job performance.* 

Bishop suggests that one reason for the poor relationship 
between grades and job performance at the high school level 
is that employers do not receive transcripts or similar 
information about school performance and would not be sure 
how to interpret such infornuUion if they did have it. This 
does not appear to be the case for college grades, however. 
Anecdotal evidence strongly suggests that college recruiters 
can gel college transcripts and do understand them, espe- 
cially within professional fields such as engineering and 
business. 

One hypothesis, drawn from ihc fact that «':cdes are poor 
predictors of job performance, is that c()lleg(^ performance* is 
larg(4y irrelevant to performance iti th*- workf)lacf/ An 
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alternative hypothesis is that elements of college 
performance are relevant and important, but grades are not 
the relevant indicator. But the fact that grades do function 
better in predicting performance as the links between 
education and job content become closer (e.g., in graduate 
and professional programs) indicates that grades, as 
currently designed, can be revealing in the light 
circumstances.'* Furthermore, it may be that other aspects of 
college education not usually considered by policymakers 
mav also be relevant to work life. Howard (1986) found, for 
example, that extracurricular experiences are much better 
predictors of work performance than are grades. 

Whatever the reason, the fact that grades do not predict 
performance means that employers will not rely on them 
when making hiring decisions. .\nd if good grades do not 
""pav off," students who have no plans to enter graduate 
school will have even less incentive to work hard in their 
classes. Employers are also harmed. They must rely on 
expensive alternatives to predict job performance; the 
greater the error in predicting which graduates will excel in 
a position, the greater the cost to the economy in terms of 
perform.Mice. In practical terms, in order to strengthen the 
coiHiection between education and work, we must find 
assessment devices for student achievement that both 
measure performance in college and predict performance on 
the job. 

There is another important reason for being concerned 
about strengthening the relationship between college 
achievement and job [)erformance. The Supreme ( ourt, in 
cases such as Griggs v Duke Pou er, found that many criteria 
used lo select emplovees w(Te based on factors that were not 
closely associated with job performance. The use of such 
criteria had a disparate* impact on protected groups in 
soci(*ty, such as racial minorities, and therefore was 
unconstitulional (Miiployinent discrimination. In other 
words, unless selection criteria can be shown to be valid 
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predictors of actual job performance, any employer who uses 
them in employment decisions mns a great risk of being 
successfully prosecuted for discrimination. Given the poor 
relationship between grades and job performance, an 
employer who bases employment decisions on grades is 
asking for legal trouble — even an employer whose noble 



goal is only to use job offers as a device to encourage 
students to work hard in school. The only way to expect 
employers to use measurements of student performance in 
hiring decisions is to produce school performance 
measurements that relate to job performance. 



III. What Can We Learn From Industry Practices? 



If college grades do not predict job performance, what 
does? Selection tests are one effort to identify and establish 
those characteristics of applicants that predict future job 
success. The selection tests' ability to establish what is 
important for job success may offer some general 
recommendations for developing tests to assess college 
performance. 

In this context, selection t(*sts face a difficult mission 
because it is often hard to define, let a\one measuitr, 
"superior job performance,'* and the asjX'Cts of performance 
that lie outside the control of individuals are often powerful 
in their effects. For emplovers, any improvement over 
chance counts as a big success, especially when the costs of 
the tests are minimal,* ^ylost selection ;.)rocedures do not 
explicitly test college classroom experiencrs, with the 
exception of jobs that require occur^at'on-specific skills, 
such as accounting. 

The selection procedures that make the most explicit use 
of maierial associated with cc'depe education are described 
in the following subsections/ 

/ 

/ 
/ 

/ 

/ 
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Ability Tests 

Ability tests are designed to assess how much one has 
learned about a particular subject or area. They can be 
subdivided into (1) achievement tests ^ which focus on 
organized learning (typically classroom instruction of 
paradigmatic material); and (2) aptitude tests, which focus 
more on informal experiences and information, "Aptitude'' 
is a prediction about future learning, and the argument 
underlying these tests is that learning is easier when built 
on a base of even informal information. 

The best known and most widely used ability test is the 
General Abilities Test Battery (GATBy), which may serve to 
illustrate the standard components of these tests. 
Developed by the U.S, Employment Service as a screening 
device. GATBy measures nine basic aptitudes; ihe first 
three (the "cognitive composite") are thought to be the most 
closely associated with college-level jobs: intelligence 
(general learning ability), verbal aptitude, and numerical 
aptitude (arithmetic). Tests similar to these are often based 
on academic classroom material and are correlated with 
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academic achievement. Gottfredson (in Ash et al. 1989) 
found a correlation of 0.60 between general intelligence/ 
mental aptitude srores and level of educational 
achievement. 

What is striking about ability tests is that although they 
are related closely to material that is taught in the 
classroom, they are typically three or four times better at 
predicting job performance than course grades, which 
(presumably) are the more direct measure of classroom 
performance!" One possible explanation for this contradic- 
tion is that the grades typically available to employers are 
not broken out in ways that would reveal performance on any 
of the test items. Verbal aptitude may be measured by the 
"class participation" component of grades in several 
courses, but there is no way to parcel out an overall 
assessment of verbal ability from course grades. Overall 
grade point averages, on which most studies rely, mask 
achievement in specific courses in which job-relevant 
knowledge, skills, and abilities (KSAs) may be measured. 
Performance in applied math and statistics courses reveal 
job-relevant math and data skills; architecture and design 
courses may also reveal a great deal about spatial relations, 
a key predictor of performance in many occupations. 

Bio-Data 

Information about a job candidate's background is often 
referred to as bio-data. The theoretical arguments in 
support of bio-data as a selection device are rooted in 
notions of consistency: past performance predicts future 
performance. Overall, bio-data are thought by many to be 
the best predictors of job performance available.*^ 
Sometimes, the relationships with performance are straight- 
forward, such as when success in an engineering internship 
program predicts success as an engineer; but other times, 
they are less obvious. One of the best-known anecdotes in 
this field is thai the question "Did you ever build a model 



airplane that dew?" predicted success in flight training 
almost as well as the entire battery of aptitude tests adminis- 
tered by the U.S. Air Force during Worid War II. 

What bio-data offers is the possibility of assessing KSAs 
that have been acquired outside of traditional classroom 
settings or even those KSAs that are acquired in the 
classroom but not measured by grades. For most adults, 
college represents an important, fundamental period of 
development, and it would be remarkable if life experiences 
during that period did not explain something about later job 
performance. Howard (1986) found, for example, that one's 
major subject in college was the best predictor of job 
performance ai AT&T; the next best predictor was participa- 
tion in more extracurricular activities, with more leadership 
positions in those activities. 

Work Samples 

The idea behind work samples is straightforward: to 
assess whether someone will perform well as a typist — give 
them something to type. There is little doubt that work 
samples have the strongest conceptual validity of any 
selection device because of their clear point-to-point 
consistency. Asher and Sciarrino (1974) found that in terms 
of their validity, work samples are a close second to bio- 
data. Schmitt and colleagues (1984) found strong support 
for using work samples in their meta-analysis of selection 
devices. "Assessment centers," a selection method using 
multiple simulations of real work problems ("in-basket 
tests" are one popular component), can be thought of as 
using work sample methods tailored specifically to 
managerial jobs. "Occupational competency tests," 
typically paper-and-pencil tests that attempt to get al work 
sample issues, also do a good job of predicting work 
performance. Hunter's (1983) meta-analysis suggests that 
occupational competency tests are twice as effective as 
ability tests in predicting jub performance. 
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Perhaps the main drawback to using work samples is that 
it is difficult to design a work sample that could assess the 
entire range of tasks that a complex job might entail — and 
assessment center exercises that duplicate this broad range 
of tasks may be expensive. Work samples also have 
problems in cases in which candidates need more than 
entry-level KSAs to perfonn the job. 

Summary 

Ability tests, bio-data, and work samples all offer some 
suggestions f '^r improving assessment of college student 
performance. If classroom grades could be broken down to 
reveal performance in areas such as verbal ability or 
memory, they should be indicative of subsequent job 
pei^ormance because they would essentially duplicate 
ability tests. 

Detailed information on extra-curricular activities (not 
only membership, but also duties performed, positions held. 



etc.) are bio-data that may reveal a great deal about the 
KSAs acquired by a student; these bio-data should be 
acknowledged more formally by colleges as part of a 
student's education, s^^^ 

Finally, many college expenert<:es, both in and out of the 
classroom, provide what is in Offence the equivalent of work 
samples tests. Completing a long research paper, for 
instance, reveals a great deal about organizational skills, 
ability to communicate through writing, and knowledge of 
problem-solving. The ability to run a laboratory' experiment 
from start to finish may measure ability to maintain self- 
discipline over a long period, level of attention to detail, and 
ability to analyze data. To get some sense of how rich the 
information about college experiences could be, consider 
what it would cost to obtain some of the same information 
from assessment centers. Indeed, college can be thought of 
as a 24-hour-a-day assessment center exercise. 



IV. Job Analysis 



The other way to get at the question of what predicts job 
success is to look directly at the requirements of jobs. Job 
analysis refers to systematic efforts to collect information 
about the work requirements associated with particular jobs. 
The biggest boost given to this analytic approach came from 
the court cases that tested the validity of selection 
procedures for employment. The court decisions required 
that selection be based on actual job requirements, which 



forced employers to introduce job analyses to determine job 
content 

Job analyses are really just detailed frameworks for 
describing jobs. Generally, the same frameworks are used 
to assess jobs that are filled by high school and college 
graduates. The various methods of job analyses can be 
divided into two broad categories: one focuses its 
descriptions on the job and the tasks performed, and the 
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other is written from the perspective of the worker and 
describes the KSAs required. The latter is clearly the more 
useful for the purposes at hand because it describes what 
jobs demand from workers. 

Appendix A describes some of the most widely-used job 
analysis systems and identifies the KSAs that are stressed in 
them. Several requirements are present .i irtually ever)- 
system of job analysis: 

1. Interpersonal skills 

2. Communication (both oral and written) 

3. Critical thinking, broadly defined (problem 
solving, reasoning, and so forth) 

4. Motivation and other personal attitudinal 
characteristics 

5. Working with data and information 

6. Math skills 

The Secretary's Commission on Achieving Necessar}' 
Skills (SCANS) at the U.S. Department of Labor has 
attempted a similar exercise by identifying the KSAs that 
are demanded by entr\'-level jobs in the current economy. 
The goal was to use these generic job requirements to help 
develop educational curricula. Se SCANS report really 
amounted to a public-policy-based job analysis and was able 
to identify entry level job competencies using the following 
"'^foundations'': basic skills (reading, writing, math, 
listening, and speaking); thinking skills (creative thinking, 
decision making, probleiii solving, visualizing symbols, 
reasoning, and knowing how to learn); and personal qualities 
(responsibility, self-esteem, sociability, self- management, 
and integrity). This list is not dissimilar to the conclusions 
gained from the job analysis literature. 

job analyses capture what is currently required by jobs 
and not what will be required in the future. Studies such as 
Workforce 2000 (1984) argue that the distribution of jobs in 
the economy is shifting away from low-skill positions (e.g.. 



manual labor) and toward higher-skill jobs (e.g., 
engineering). While there is likely to be a shift in this 
direction, new research suggests that the rate of change will 
be no greater than in past generations (Cappelli 1991). 

Studies that look at changes in current y hs (e.g., how 
management jobs may differ in the future) are more 
speculative and are driven mainly by assumptions about how- 
organizations may change. Porter and McKibbin (1988) 
conducted a study for the American Assembly of Collegiate 
Schools of Business. Their conclusions, from extensive 
inter '^nvs, suggest that education needs to be more applied, 
helping students see the links to practice, and that 
interpersonal and leadership skills should be emphasized 
and oriented toward managing people. The SEI Center at 
the Wharton School conducted a similar study to serve as 
the basis for designing a new business school curriculum. 
Their recommendations included more extensive training in 
interpersonal skills, greater integration across disciplines, 
and more breadth in education (Wind and W^est [in press]). 

Interviews with human resource consultants in firms that 
specialize in job analyses show a clear consensus opinion 
that "flatter" organizations (those with less hierarchy) are 
forcing employees to be more autonomous.'" The reduction 
in stnicture and control associated with this change implies 
grec^ter reliance on leadership skills as an alternative to 
managing employees. Communication skills are also 
becoming more important as employees are assigned more 
informal reporting arrangements that involve more people, 
and as matrix organizational structures and team methods of 
work organization force employees to work moir with each 
other. In general, interpersonal skills become more 
important as working in teams becomes more prevalent. The 
ability to he flexible and adapt to new circumstances is 
another general theme that is driven by the continuing 
turl)ulence in modern corporations. 
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While these speculations suggest that job requirements in 
the future should place more emphasis on interpersonal 
skills, the basic set of KSAs will probably differ little from 
the above list. 

It should come as a great relief to those concerned about 
the potential dominance of education by business to learn 
that the general KSAs that are identified as necessary for 
most jobs are not narrow sets of job-specific issues. They 
are, in fact, basic educational competencies that are 
compatible with many other objectives of education. More 
importantly, most are already taught in college, albeit some 
indirectly. Math skills are developed in math classes and in 
courses that use applied math. Critical thinking is taught 
explicitly in logic courses and should be a part of a broad 
array of courses from history to the social and behavioral 
sciences. Working with data is explicit in statistics courses 
and in all branches of the sciences that use applied statis- 
tics. Communications should be a part of ever)' course that 
requires discussion and writing. Interpersonal skills are the 
subject matter of some courses (e.g., negotiations and group 
dynamics) and can be developed in any course that requires 
team projects and in extracurricular programs, such as team 
sports and organizations. Motivation and personal 
characteristics, such as integrity, are no doubt the least 
likely to be taught in a classroom context (although mililary 
and religious schools make explicit attempts to develop 
these characteristics through socialization; and extracurricu- 
lar activities, such as athletics, may develop them as well). 

Suggestions about how these sets of KSAs, which are 
required in employment, could be developed m.ore 
thoroughly in college instruction do not need to be 
revolutionary. Courses in any subject in which students are 
required to write papers, discuss material, and work in 
groups go a long way toward developing many of the targeted 
KSAs. When such courses challenge students to analyze 
problems and think critically about them, we are more than 



halfway toward completing the list. Courses that make use 
of math concepts and data teach students how to apply these 
factors to problems. Such instruction is contained in an 
increasingly large proportion of college curricula — math and 
statistics, all the sciences (natural, behavioral, and social), 
and often history and anthropology as veil. 

With this description, it becomes easy to see why grades 
may not be good predictors of job performance, even for 
subjects with job-relevant course materially. Consider a 
hypothetical course in which the material is related directly 
to employment (such as human behavior), which is taught in 
a large lecture format in which students talk neither with the 
instructor nor with each other, and which stresses memoriza- 
tion of the results of prior research. Few job-related skills 
will be developed in the process of presenting the course 
material. Multiple-choice tests, which typically are used for 
grading in such courses, would not reveal a student's ability 
with these skills, even if they had been developed. 

Now, consider the same course taught in a small-group- 
discussion format with students doing at least some of their 
work in teams; with the material requiring students to apply 
theories and statistical methods to real-life problems; and 
with grades based on written efforts to evaluate course 
material and on class participation. In the latter situation, 
the education process develops many useful skills, and the 
grading procedure can evaluate them. Reports of a recent 
five-year study at Harvard found that students actually 
preferred courses with more written assignments and that 
the. performed better when assignments required working in 
groups. 
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V. Conclusions on Improving College Assessments 



The fact that grades are poor predictors of future job 
performance is a cause for concern. Part of the problem 
may be that college is simply not teaching the skills for job 
success, which is especially worrisome because the KSAs 
necessary for jobs appear to coincide with basic attributes 
needed in all aspects of life. The good news in this area is 
that the changes in instruction that will lead to developing 
important KSAs are straightforward and can be 
accommodated within existing curricula. 

The more important and immediate issue, however, is 
with assessments. Course grades, the main method for 
assessing student performance, cannot measure many of the 
"work-world-relevant" .skills that a college education 
already does provide, f'ortunately, experiments in 
alternative assessment methods are underway. The 
University of Massachusetts at Boston, for example, requires 
students to complete an essay exercise at the end of their 
four-year program, in order to judge their critical thinking 
abilities. Alverno College in Minnesota may have the most 
elaborate alternative system. Using assessment center 
simulations that were designed with help from AT&T's 
personnel department, Alverno evaluates its students' job- 
related skills as identified from studies of successful 
practitioners. 

Alternative systems of assessment do not need to be 
nearly so ambitious. The greatest improvements could be 
made by simply assembling existing information about 
student performance in more innovative ways. For example, 
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an overall measure of performance on written material, such 
as an average grade for all research papers produced by a 
?»udent during college, would reveal a great deal about 
written communication skills as well as problem-solving 
abilities and creativity. Such an indicator requires no new 
data to construct. The University of Michigan, the College 
of William and Mary in Virginia, and other colleges have 
already compiled portfolios of student work over their entire 
program, a much more difficult task than simply adding up 
grades. A summary class participation grade across^all 
courses would reveal something similar about verbal 
communication skills. 

Significant student projects, such as semester-long 
laboratory experiments or group projects, are work samples 
with direct relevance to job performance, and the assess- 
ment of these efforts could be included as part of a student's 
overall record. Employers already solicit bio-data from 
students in the form of job applications, although the 
accuracy of this self-reported information is often 
questionable. Colleges could help ensure the accuracy of 
information about school-related extracurricular activities 
by reporting it as part of a student's record and adding 
details (e.g., positions held, duties performed, etc.) to 
highlight its significance. "Project Worklink," run for high 
schools by the Educational Testing Service, summarizes a 
student's extracurricular activities and prior work 
experience and offers transcript data in an effort to provide 
job-relevant bio-data. 
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There is even' reason for believing that employers would 
jump at the chance to make use of this repackaged 
assessment information and that any school providing better 
job-relevant assessments would have a great competitive 
advantage in securing employment for its students (at least 
for those students who perform well!). One of the main 
reasons given by employers for establishing close ties to 
professional schools is simply to become familiar with 
courses, faculty, and extracurricular programs in order to 
understand the context of student records and identify the 
best performers. The experience of the Wharton School at 
the University of Pennsylvania indicates clearly, for 
example, that employers are interested in close ties — and in 
donating money — where they believe that those lies will 
help them to recruit at the school. 



We appear to be headed toward the development of a 
national system for assessing college student performance, a 
potentially tortuous exercise in which ultimately the goal 
may become as much to monitor colleges as students," 
Strengthening the link between education and work, 
however, requires developing assessments of student 
performance that also predict job performance. It is 
unlikely that any national tests will be able to provide as 
much information about a student's job-relevant skills as 
could be provided by a simple repackaging of existing 
assessments and exj)eriences. The benefits of more job- 
relevant assessments — as a signal to employers that will 
provide incentives for students to achieve, as a reduction in 
the selection costs for employers, as a way of strengthening 
relations between colleges and employers — are important 
enough for colleges to begin making changes now. 
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Job Analysis Systems and KSAs 

Hay Associates Profile System: Tfie Hay Group is a 
large compensation consulting firm that performs job 
analyses on jobs covering some 2 million workers in the 
.United States. Its job analyses focus on three areas: 

1. knoiV-liOiV — concerns the techniques 
and procedures required by jobs. Examples 
of know-how are professional skills, such as 
accounting or engineering, and general 
management skills, such as designing plans. 
More difficult jobs have associated 
specialized and technical skills and greater 
breadth required across skills. 

2. Problem solving — refers to the thinking 
demands made by jobs. Routine, repetitiv(^ 
tasks fall at the lower end of this s(^ale. w hih^ 
those that have only abstract definitions, 
requiring adaptive abilities, fall at the upper 
end. 

3. Acrnuutahility — refers to the freedom to 
act that is accorded to employees in a job. 
Jobs that offer employees litth^ guidance and 
that also are associated with the potential to 
have a larg(^ impact on the organization score 
high on this scale. 

The Position Analysis Qnestionnaire (PAQ): 'Yhc PAQ 
has been the most thoroughly resea died and acad(*micaliy 
prominent of the job analysis nuHhods.'^' The th(Mii(^ of the 
PAQ is to identify the basic behavior and aptitude 
requirements of jobs. The 187 items in the (juestionnaire 
can be divided into six general categories: 

1. Information: vhere and how oik* gets 
information n(*e(led fortlu^job 

2. Menial processes: reasoning, dc^'ision 
making, and so forth 



3. Work output: physical a(^tivities, tools, 
and so forth 

4. Relationships ii ith others: measures of 
complexity 

5. Job context: social and physical context 
of work 

6. Other: a "catch-all" category- 



Although the PAQ focuses on work behaviors as opposed 
to tasks and has sometimes been criticized in the context of 
differentiating jobs, it helps to identify what workers need to 
know across a broad range of jobs that require similar kinds 
of behavior. 

The ability of the PAQ to identify basic work KSAs has 
been examined through a series of tests of the relalionship 
betw een PAQ job scores and the performance of job holders 
on the (iA TBy, which is perhajjs the most widely used test of 
employment aptitude. The idea is that people gravitate 
toward jobs that us<» their skills; so these studies correlated 
GATBy scores of incumbents with PAQ scores fo^ their jobs. 
McCormick and Jeanneret (1988. 831) sunmiarized the 
results, which are strong.'"* 

Private firm studies using commercial tests of 
intelligence and verbal, numerical, spatial, and clerical 
aptitude also show reasonably good correlations with the 
PAQ (about 0.70). What these tests show is that PAQ 
nu-asures of job requirements track the characteristics held 
by workers in those jobs. This is not the same as 
establishing validity — identifying "true" requirements of 
jobs — but these results are consistent with a valid measure 
under the assumption tiiat workers sort themselves out by 
job according to KSAs. 
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The Management Position Description Question- 
naire: This questionnaire was developed by Control Data 
Business Advisors for use with their own managerial 
employees. It has, however, become popular in many white- 
collar organizations, in part because its focus on managerial 
jobs made it appear more applicable to this type of 
business.'** The basic KSAs can be categorized as follows: 

1. Uadership skills: motivation, coaching 

2. Administrative skills: planning, 
allocating 

3. Interpersonal skills: conflict 
management, group process skills 

4. Communications and decision making: 
informtion management, analytic 
ability 

5. Professional knowledge: company- 
specific practices, technical skills such 
as accounting 

The Threshold Traits Analysis System; This system is 
a different approach that focuses explicitly on individual job 
holders, rather than on the jobs themselves, and examines 
their personality traits.''^ The traits can be broken down into 
ability factors (which are subdivided into aptitude for 
acquiring knowledge or skill and proficienc\ in skills 
alreadv possessed) and attitudinal factors (which affect the 
willingness to perform at given levels). The specific traits 
arc categorized as follows: 

1. Physical traits, such as strength 

2. Mental traits, such as problem-solving 
and memory 

3. Ijcarned knowledge and skills, such as 
r<>mmunication, motivation, and 
adaptability 

4. Social traits, such as influence and 
cooperation 
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Ability Reqiurement Scales: The Ability Requirement 
Scales attempt to identify generic abilities and are based on 
50 item categories. Perhaps more than any of the other job 
analysis sysfems described here, the Ability Requirement 
Scales focus on physical and perceptual factors. 
Communication skills, reasoning, and problem solving 
feature heavily among the nonphysical and perceptual 
factors.'^' 

Functional Job Analysis: This .-node of analysis was 
developed out of the need to determine the worker charac- 
teristics required for jobs described in the Dictionary of 
Occupational Titles. The method is designed to be 
straightforward, and job analysts are required to make far 
fewer decisions on their own than is the case in most 
methods. The scales used in the Functional Job Analysis 
fall into six categories: 

1. Data functions: complexity in the use 
of information 

2. People functions: level of interpersonal 
skills demanded 

3. Functions that involve u.sing objects: 
phvsical requirements, typically with 
machines 

4. Worker instructions: le\el of 
responsibility 

5. Reasoning development: from common 
sense to abstract undertakings 

6. Mathematical development: math skills 

7. Writing functions^' 

The Secretary's Commission on Achieving Necessary 
Skills (SCANS): The SCANS is a public policy . tudy of the 
particular KSAs demandt^d by jobs in the current economy. 
The Commission was charged with identifying the 
requirements for entry-level jobs, with the idea that once 
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generic job requirements were identified, they coukl then be 
used to help shape what is taught in schools. The SCANS 
report really amounts to a public-policy-based job analysis. 

Endnotes 

' For example. Maeroff ( 1991 ). 

^ The rationale for this situation is the difficulty of (lra\sinp any 

conrluj>i<»ns about the absence of significant effects because of the 
many problems that can mask significant results. 

Some earlier meta-analyses of grades have shown relati()n>hips 
between gra<le.s and performance, albeit >mall one>. l'roponent> of 
using grades point out that even small validity coefficients may be 
economically useful. Assuming that the standard deviation of 
performance in a particular job is equivalent to SlO.OOO per year 
(reasonable for management jobs), a validity coefficient of O.U) for 
grades implies an improvement over chance of SI, 000 per year in 
performance when u>ing grades a> a selection device, a substantial 
gain when measured as pre^ent discounted value. Statistical 
corrections made in recognition that performance criteria are often 
uncertain and that the range of performance is restricted by the 
selection pro<*ess (i.e., those hired may be more similar than the 
overall pool of applicants) can change the validity coefTicient> 
substantially, often raising them. The American Psychological 
.Asswiation suggests that these corrections be rep()rted along with the 
regular validity coefficients. On the other hand, these validity 
studies of gra<ies typically do not report what marginal gain result> 
from using grades as one of several predictors to detennine whether 
even the small predictive power of grades is, in fact, the result of 
some other, confounding factor. It is possible, for example, that effort 
is what really matters for job performance, and grades are simply a 
proxy — a poor proxy — for effort. 

* Klilgard (1985) has prwiuced a review that concentrates on studies in 
sociology, which find thai student performance explains bow students 
get sorted in: . m-cupations, but that such performance explains lill!»- 
about the success within occupation^. SeeTaubman (August 19^)1) 
for a summarv" of recent changes in the return to higher education. 

' There is some evidence for this \iew at the high sch(H)l le\el. Bi>hop 
(1991) finds, for example, that higher levels of competency in math 
reasoning, verbal, and science abilities (as measured by the Armed 
Services Vof ational Aptitude Ratters) actually received a negative 
reward from the labor market in terms of wages. The most plausible 
explanation for this result is that these competencies were not 
required by most jobs held by non-college-bound youths. 



Dye and Reek's (1989) suney found that grades in one's major field 
are better predictors; Bretz's (1989) suney found that grades explain 
fjerfomiance belter in business and education, in which students are 
more likely to have received training in programs specific to those 
fields. Howard (1986) found that graduate grades are better 
predictors than undergiaduate grades and that grades in a specific 
business program (e.g., MB.\) are better than masters degrees taken 
as a group. Weinstein and Srinivasan (1974) also found relationships 
l>etween grades in MBA programs and later salaries, 

■ The payoff of a given selection test \aries by employer and depends on 
the characteristics of selection in the absence of the test, the '*base 
rate" (percentage of employees who succeed), anil the "selection 
ratio" (percentage of applicantj. hired). Exf)ectancy charts calculate 
the relum> for a given lest a< cording to its validity and the 
employer's base and selection latios. In general, tests offer the 
highest payoff when both ratios are around 50 percent. 

" See Reilly and Chao (1982) for the relative perfonnance of grades and 
Barrett and Depinet ( 1991) for the most successful ability test results. 

" F(jr example. Owens (1976) and Schneider ( 1976). .\sher (1972) 
presented summarv data suggesting that by some criteria, bio-data 
were almoM twice as successful at predicting job performance as 
intelligence aptitude, the next best method examined. Sparks' (1983) 
data indicate that bio-data are far and aw ay the best predictors of job 
proficiencv. McDaniel. Schmidt, and Hunter's (1988) meta-analysis 
found that prior work experience is the best predictor of job 
performance. Bishop (1991 ) concluded that the previous five year's 
work experience has a bigger impact on productivity than five grade 
levels of math and verbal ability. This is not to say, however, that 
bio-data ha\e not been criticized. Koniian (1966) found that bio-data 
have real problems predicting managerial perlormance and may be 
inferior lo other methwls for such jobs. 

Our thanks to Marsha Cameron at the )X'yatt Company, Joy Hazucha at 
Personnel Decisions, Inc, Charles Lee at TPF&C (Towers Perrin), 
,Andy Rosen at Hay .Associates, and Eugene R, Smoley, Jr,. at Cresap 
(Towers Perrin) for their thought> on the changes in managerial jobs. 

" See Madaus (1991 ) for a discus>ion of some of the problems 
assm'iated with national tests. 

'* McConnick. Kmest J. and P. Richard Jeanneret. 1988. "Position 
Analvsis Questionnaire," Sidney Gael, ed. The Job A mil} sis 
Handbn'ik for Business, Industry, and Gorernnienl. New York: John 
W iiey and Sonj>, 82,5-842. There is als(j a Professional and 
Managerial Position Questionnaire (PMPQ), which is ver\ >imilar. 

The a>sumpti{m is that on average, current incumbents in jobs have 
the exact KSAs nere>sar\ to do their jobs — that there are no 
overcjualifit'd or under-qualified workers. This may l>e a reasonable 
a>>umption as l(»ng as the job analysis i> basH on a large nunilier of 
incumbents. At the ven least, the match between KSAs and 
requirements cannot \ar\ acro>>jobs. 
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